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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 03 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period wilt apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)KI This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-29 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-29 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)D All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 
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1 ) S Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). 

2) CD Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) d Notice of Informal Patent Application (PTO-152) 

3) Information Disclosure Statement(s) (PTO-1449) Paper No(s) 5 . 6) □ Other: 
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DETAILED ACTION 



Information Disclosure Statement 
1 . The information disclosure statement (IDS) submitted on 03/12/02 has been considered. 



2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AEPA) and the Intellectual Property and High Technology Technical Amendments Act of 2002 
do not apply when the reference is a U.S. patent resulting directly or indirectly from an 
international application filed before November 29, 2000. Therefore, the prior art date of the 
reference is determined under 35 U.S.C. 102(e) prior to the amendment by the AIPA (pre-AIPA 
35 U.S.C. 102(e)). 

3. Claims 1-29 are rejected under 35 U.S.C. 102(e) as being anticipated by Shimizu (US 
5,796,991). 

Re claim 1, Shimizu discloses an image processing apparatus comprising: 
an image processing part (105 of fig. 7; see also 211 and 213 of figs. 3, 5 and 7 in 
details) for receiving images captured by a plurality of cameras (201L and 201R of fig. 7) 
shooting surroundings of a vehicle to generate a synthetic image (213 of fig. 7) viewed from a 



Claim Rejections - 35 USC § 102 
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virtual point of view from these camera images, wherein the image processing part changes at 
least one selected from a position, a direction of a line of sight, and a focal length of the virtual 
point of view in accordance with a running state of the vehicle (col. 5, lines 9-61). 

Re claim 2, Shimizu further discloses wherein the image processing part changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
of view in accordance with a running speed of the vehicle (232 of figs. 5 and 7). 

Re claim 3, Shimizu further discloses wherein the image processing part changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
of view, and controls capturing of an image outside a view range of the changed virtual point of 
view (111, 230, 241 and 242 of figs. 5 and 7; see also col. 7, lines 43-51; col. 8, lines 57-61, col. 
9, lines 49-52, col. 10, lines 16-58). 

Re claim 4, Shimizu further discloses wherein the image processing part (230 of figs. 5 
and 7) controls the capturing of an image outside a view range of the changed virtual point of 
view by changing a model for image synthesis (245 of figs. 5 and 7, see also col. 10, line 44 
through col. 11, line 11). 

Re claim 5, Shimizu further discloses wherein the image processing part changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
of view in accordance with a steering angle of the vehicle (231 of fig. 7). 

Re claim 6, Shimizu further discloses wherein the vehicle includes an object detection 
sensor for detecting an obstacle (201 of fig. 7), and the image processing part changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
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of view in accordance with results of detection by the object detecting sensor (col. 11, line 56 
through col. 12, line 10). 

Re claim 7, Shimizu further discloses wherein the image processing part includes an 
original mapping table (21 1 and 212 of figs. 5 and 7) and generates a synthetic image using a 
mapping table that is cut out from the original mapping table, and the image processing part 
changes at least one selected from a position, a direction of a line of sight, and a focal length of 
the virtual point of view by changing the mapping table to be cut out from the original mapping 
table (col. 6, lines 33-67, wherein the mapping table is a correspondence relationship between 
pixel, each of pixels of mask image, for a synthetic (213 of figs. 5 and 7). 

Re claim 8, Shimizu further discloses an image processing apparatus comprising (fig. 7) 
an image processing (213 of fig. 7) part for receiving images captured by a plurality of cameras 
(20 1L and 20 1R of fig. 7) shooting surroundings of a vehicle to generate a synthetic image 
viewed from a virtual point of view from these camera images, wherein the image processing 
part controls capturing of an image outside a view range of the virtual point of view in 
accordance with a running state of the vehicle (232 of fig. 7). 

Re claim 9, Shimizu further discloses a monitoring system comprising: 

a plurality of cameras (20 1L and 20 1R of fig. 7) shooting surroundings of a vehicle; an 
image processing part for receiving images captured by the plurality of cameras to generate a 
synthetic image viewed from a virtual point of view from these camera images; 

a display part (102 of fig. 7) for displaying the synthetic image, wherein the image 
processing part changes at least one selected from a position, a direction of a line of sight, and a 
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focal length of the virtual point of view in accordance with a running state of the vehicle (232 of 
fig. 7). 

Re claims 10, 16, Shimizu further discloses an image processing apparatus comprising: 

an image processing part (211 of fig. 7) for receiving images captured by a plurality of 
cameras (201 L and 20 1R of fig. 7) shooting surroundings of a vehicle to generate a synthetic 
image (213 of fig. 7) from these camera images, 

wherein the image processing part (21 1 of fig. 7) generates an image including a first 
image (20 1L of fig. 7, e.g. the image is captured by the left camera) and a second image (20 1R 
of fig. 7, e.g. the image is captured by the right camera) as the synthetic image (the output of the 
image synthesizing unit (213 of fig. 7), the first image being viewed from the virtual point of 
view (HMD 102 of fig. 7), 

the second image being viewed from a viewpoint that is different from the virtual point of 
view of the first image in at least one selected from a position, a direction of a line of sight and a 
focal length, or the second image being different from the first image in a model (245 of fig. 7, 
e.g. CG image forming unit generates a model image for viewing). 

Re claim 11, Shimizu further discloses wherein the second image is at least one of the 
camera images (20 1L and 20 1R of fig. 7). 

Re claim 12, Shimizu further discloses wherein the first image is a close view image 
showing the vehicle and surroundings thereof, and the second image is a distant view image 
showing an area distant from the surrounding area of the vehicle that is shown by the close view 
image (fig. 6 A, fig. 10A-10E). 
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Re claims 13, Shimizu further discloses wherein the image processing part arranges the 
distant view image around the close view image in the synthetic image (figs. 6A and 6B). 

Re claim 14, Shimizu further discloses wherein the distant view image is an image 
having continuity with the close view image (figs. 10A-10E). 

Re claim 15, Shimizu further discloses wherein the first image shows at least a part of the 
vehicle and at least a part of the surroundings of the vehicle, and the second image is obtained by 
enlarging at least a part of the region shown by the first image (figs. 10A-10E). 

Re claims 17 and 23, Shimizu further discloses an image processing apparatus 
comprising: 

an image processing part (21 1 of fig. 7) for receiving images captured by a plurality of 
cameras (20 1L and 20 1R of fig. 7) shooting surroundings of a vehicle to generate a synthetic 
image (213 of fig. 7) from these camera images, wherein in the synthetic image, the image 
processing part displays at least a part of a vehicle region where the vehicle is present, and an 
attention drawing region for drawing attention in which at least a part of the surroundings of the 
vehicle is shown (fig. 3). 

Re claim 18, Shimizu further discloses wherein the synthetic image is an image viewed 
from a virtual point of view that is set above the vehicle (figs. 10A-10E). 

Re claim 19, Shimizu further discloses wherein the image processing part displays an 
illustration image or an actual image of the vehicle on the vehicle region (203 of fig. 3, e.g. the 
display (203) can display an actual image of the object or vehicle). 

Re claims 20-21, Shimizu further discloses wherein the attention drawing region includes 
at least a part of a blind spot region around the vehicle that is not shot by any of the cameras 
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(20 1L, 20 1R, and 21 1 of fig. 7, e.g. the image processing unit (211) can process the images 
picked up by the cameras, wherein image includes the region having at least a part of blind spot 
regions around). 

Re claim 22, Shimizu further discloses wherein the image processing part determines a 
range of a region obtained by combining the blind spot region and the vehicle region, using 
region data showing a projection region of the vehicle in each camera image (201 of fig. 5 or 
103 of fig. 2, e.g. wherein image pickup means has two image pickup devices 201L and 201R 
which pickup images over a range corresponding to the field of view to be presented to the left 
eye and the right eye, respectively, of the user) 

Re claim 24, Shimizu disclose an image processing apparatus comprising: 
an image processing part (21 1 of fig. 7) for receiving images captured by a plurality of 
cameras (20 1L and 20 1R of fig. 7) shooting surroundings of a vehicle to generate a synthetic 
image (213 of fig. 7) from these camera images, 

wherein the image processing part (213 of fig. 7) generates the synthetic image, 
using a mapping table (212 of fig. 7; e.g. the mapping table is the relationship correspond 
between pixels) including first mapping data describing a correspondence relationship between 
pixels of the synthetic image and pixels of the camera images (Left image (20 1L) and Right 
image (201R), 

and second mapping data describing an identifier showing that a pixel of the synthetic 
image corresponds to pixel data other than the camera images (243, 244, and 245 of fig. 7, e.g. 
stored data from CG MODEL data base (244 of fig. 7)). 
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Re claim 25, Shimizu further discloses wherein the pixel data other than the camera 
images show the vehicle or a blind spot region that is present in at least a part of the 
surroundings of the vehicle (201R and 201L of fig. 7 , e.g. the camera is able to capture the blind 
spot region). 

Re claims 26 and 27, Shimizu further discloses wherein the image processing part stores 
a predetermined image other than the camera images (CG MODEL DATABASE, 244 of fig. 7), 
and with respect to the pixel of the synthetic image (213 of fig. 7), the second mapping data 
describe coordinate values corresponding to the pixel in the stored predetermined image (CG 
IMAGE FORMING UNIT, 235 of fig. 7). 

Re claims 28 and 29, Shimizu discloses an image processing apparatus comprising an 
image processing part (21 1 of fig. 7) for receiving images captured by a plurality of cameras 
(20 1R and 20 1L of fig. 7) shooting surroundings of a vehicle to generate a synthetic image (213 
of fig. 7) from these camera images, wherein the image processing part uses mapping data (212 
and 213 of fig. 7) describing a correspondence relationship between pixels of the synthetic image 
and a plurality of pixel data including one or both of pixel data of the camera images and pixel 
data other than the camera images, and describing a rate of necessity with respect to each of the 
pixel data, and weights each pixel data in accordance with the rate of necessity, thereby 
generating the pixel data of the pixels of the synthetic image (244 and 245 of fig. 7). 

4. Claims 1-29 are rejected under 35 U.S.C. 102(e) as being anticipated by Waters (US 
6,396,535 Bl) as shown in figures 1-3, cols. 1-4. 
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5. Claims 1-29 are rejected under 35 U.S.C. 102(e) as being anticipated by Peleg et al. (US 
66,665,003 Bl) as shown in figures 6-10. 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Hanna et al. (US 6,490,364 B2) discloses an apparatus for enhancing images using flow 
estimation. 



Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung T. Vo whose telephone number is (703) 308-5874. The 
examiner can normally be reached on 6:30 AM - 3:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris. Kelley can be reached on (703) 305-4856. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9314. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist-whose telephone number is (703) 305-4700. 



Tung T. Vo 
Examiner 
Art Unit 2613 




